Systems and methods for hand gesture control of an electronic device

ABSTRACT

Systems and methods of generating device commands based upon hand gesture commands are disclosed. An exemplary embodiment generates image information from a series of captured images, generates commands based upon hand gestures made by a user that emulate device commands generated by a remote control device, identifies a hand gesture made by the user from the received image information, determines a hand gesture command based upon the identified hand gesture, compares the determined hand gesture command with the plurality of predefined hand gesture commands to identify a corresponding matching hand gesture command from the plurality of predefined hand gesture commands, generates an emulated remote control device command based upon the identified matching hand gesture command, and controls the media device based upon the generated emulated remote control device command.

PRIORITY CLAIM

This patent application is a Continuation of U.S. Non-Provisional patentapplication Ser. No. 12/533,698, filed Jul. 31, 2009, granted as U.S.Pat. No. 8,428,368, and entitled “SYSTEMS AND METHODS FOR HAND GESTURECONTROL OF AN ELECTRONIC DEVICE,” the content of which is hereinincorporated by reference in its entirety.

BACKGROUND

Often, controlling a set top box (STB), a media device or other systemusing a remote control device is difficult or inconvenient. For example,a person eating dinner while watching a television (TV) may havedifficulty in reaching for and using the remote control device. In otherinstances, a person may have lost their remote control device and wouldhave to make changes to their media system using the manual controls onthe STB and/or the media device or system. If the person wishes toquickly perform an operation under these circumstances, such as pausingduring a scene of interest, inability to quickly access their remotecontrol device may prevent the person from performing the pause functionin a timely manner.

In some situations, it may be physically impossible for a person tooperate a remote control device. For example, a person with severephysical disabilities may not have sufficient control of their fingersto manually operate the remote control device. As another example, aperson in a hospital recovering from surgery may not be able to reachand/or operate the remote control device.

Accordingly, there is a need for a person to have alternative ways togenerate device commands, particularly if their remote control device isnot conveniently available and/or if they are unable to operate theirremote control device.

SUMMARY

Systems and methods of generating device commands based upon handgesture commands are disclosed. An exemplary embodiment generates imageinformation from a series of captured images, generates commands basedupon hand gestures made by a user that emulate device commands generatedby a remote control device, identifies a hand gesture made by the userfrom the received image information, determines a hand gesture commandbased upon the identified hand gesture, compares the determined handgesture command with the plurality of predefined hand gesture commandsto identify a corresponding matching hand gesture command from theplurality of predefined hand gesture commands, generates an emulatedremote control device command based upon the identified matching handgesture command, and controls the media device based upon the generatedemulated remote control device command.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred and alternative embodiments are described in detail below withreference to the following drawings:

FIG. 1 is a block diagram of an exemplary hand gesture control systemthat is operable to control a media device;

FIG. 2 is a block diagram of an exemplary remote control device that isoperable to control a media device;

FIGS. 3A-3M illustrate exemplary hand gestures of one of the user'shands which generate exemplary hand gesture commands; and

FIG. 4 is a block diagram of an electronic device with an embodiment ofthe hand gesture control system.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a hand gesture control system 100 that isoperable to control a media device 102, such as, but not limited to, aset top box (STB). FIG. 2 is a block diagram of an exemplary remotecontrol device 200 that is operable to control the media device 102. Thehand gesture control system 100, in an exemplary embodiment, monitorshand gestures from a user and, based upon the hand gestures, emulatescommands that may be generated from the remote control device 200.Embodiments of the hand gesture control system 100 may be implemented inother media devices, such as, but not limited to, stereos, radios,televisions (TVs), digital video disc (DVD) players, digital videorecorders (DVRs), a personal computer (PC) and/or a home appliance so asto emulate their corresponding remote control device 200.

The non-limiting exemplary media device 102 comprises a program contentstream interface 104, a processor system 106, a memory 108, a programbuffer 110, an optional digital video recorder (DVR) 112, a deviceinterface 114, an optional signal output device 116, and an optionalimage capture device 118. The memory 108 comprises portions for storingthe hand gesture recognition logic 120, the hand gesture database 122,the remote control device (RCD) command database 124, and the commandinterpretation logic 126. In some embodiments, the hand gesturerecognition logic 120 and the command interpretation logic 126 may beintegrated together, and/or may be integrated with other logic. Othermedia devices, and other devices having the hand gesture control system100 implemented therein, may include some, all, or none of theabove-described media processing components. Further, additionalcomponents not described herein may be included in alternativeembodiments.

The image capture device 118 is pointed outwardly from the media device102 in a direction that is anticipated to result in captured images ofone or both of the user's hands 130 a, 130 b. Alternatively, oradditionally, an external image capture device 132, which may beportable, may be positioned so as to be pointed towards the user 128.For example, the media device 102 may be located above, behind and/or tothe side of the user 128 such that an image capture device 118 would notbe able to capture images of the user's hand gestures. Thus, theexternal image capture device 132 may be independently placed andoriented in another location in a media room so as to capture images ofthe user's hand gestures. The external image capture device 132 maycommunicate captured image information to the media device 102 using anysuitable communication medium, including physical wires, light, or radiofrequency (RF).

The image capture devices 118, 132 are operable to capture a sequentialseries of still images of the user 128, and more particularly, images ofone or both of the user's hands 130 a, 130 b. Image informationcorresponding to the captured images is then generated and communicatedto the processor system 106. The time between successive captured imagesmust be short enough so as to provide a meaningful interpretation of themovement of the user's hands 130 a, 130 b. In some embodiments, theimage capture devices 118, 132 may be a video camera. Accordingly, imageinformation generated from a plurality of selected still images capturedby the video camera are processed to determine the hand gesturemovements of the user 128.

Conceptually describing operation of the hand gesture control system100, embodiments of the hand gesture recognition logic 120 analyze imagedata received from the image capture device 118 and/or the externalimage capture device 132. The image information is processed todetermine the location and movement of one or both of the user's hands130 a, 130 b. In an exemplary embodiment, identification of the user'shands 130 a, 130 b is based upon the identification of the location ofpredefined points on the user's hands 130 a, 130 b, known as markers orthe like. The movement of the markers are tracked to determine what handgesture the user 128 is making. A hand gesture command is generatedbased upon the tracked movement, or hand gesture, of the user 128.

The determined hand gesture command is then correlated, compared, or thelike, to corresponding hand gesture commands residing in the remotecontrol device command database 124 to identify a matching hand gesture.Then, the command interpretation logic 126 generates the appropriateemulated remote control device command(s) based upon the identifiedmatching hand gesture command.

Depending upon the determined emulated remote control device command,the determined emulated remote control device command is performed bythe media device 102. For example, but not limited to, the emulatedremote control device command may turn on or turn off the media device102.

Other emulated remote control device commands may be generated andcommunicated to other controlled media presentation devices, such as thetelevision 134. Here, the optional signal output device 116 communicatesthe emulated remote control device command to the controlled mediapresentation device by transmitting a signal that uses the samecommunication media, and the same signal format, as the remote controldevice 200. That is, the signal output device 116 may output an infraredor radio frequency (RF) signal that is identical to the correspondingoutput signal of the remote control device 200. Accordingly, based uponthe determined user hand gesture commands, the media device 102 can thencontrol various media devices in accordance with the determined handgesture commands.

For example, the user's right hand 130 a is illustrated as being held inan upward, palm-out position. Here, an embodiment of the hand gesturecontrol system 100 would interpret images of the user 128 to identifythe user's right hand 130 a, and recognize the position of the user'sright hand 130 a. Further, based upon previously captured images andusing the hand gesture recognition logic 120, embodiments determine thatthe user 128 has moved their right hand 130 a from another position (notshown) to the illustrated upward, palm-out position. In view of themovement and/or the final position of the user's right hand 130 a, thehand gesture control system 100 would understand that a hand gesturecommand has been intended by the user 128.

The identified hand gesture of the user 128 is used to generate a handgesture command. The determined hand gesture command is compared to aplurality of corresponding hand gesture commands stored in the handgesture database 122. A matching hand gesture command is identified fromthe plurality of hand gesture commands stored in the hand gesturedatabase 122. Once the corresponding matching hand gesture command inthe hand gesture database 122 is identified, then the commandinterpretation logic 126 is used to determine the emulated remotecontrol device command intended by the user 128. Then, the emulatedremote control device command controls the media device 102 and/or otherconnected electronic devices.

For example, but not limited to, a hand gesture ending with the upward,palm-out position of the user's hand 130 a illustrated in FIG. 1 may beassociated with a pause command that pauses play of a presented program.In an exemplary embodiment, a remote control device 200 includes a pausebutton 202 thereon that, when actuated by the user 128, also pauses playof the presented program. Thus, the hand gesture ending with the upward,palm-out position of the user's right hand 130 a emulates the pausebutton 202 of the remote control device 200. To pause presentation ofthe viewed program, the user 128 simply gestures with their right hand130 a by moving their hand 130 a to the upward, palm-out position.

Further, it is appreciated that the other hand 130 b of the user 128may, or may not be, moved with the intention of generating a command.Here, the user's left hand 130 b is apparently remaining in asubstantially stationary position in their lap. Accordingly, the handgesture control system 100 determines that the user's left hand 130 bhas not been moved with the intent of making a hand gesture command.

In the various embodiments, the hand gesture recognition logic 120 isconfigured to identify the hands 130 a, 130 b of the user 128 from otherparts of the user's body. For example, but not limited to, primitivesand/or artifacts corresponding to the hands and other parts of the user128 may be generated from the captured images. Thus, the user's handgestures can be distinguished from other movements of the user 128.Further, the hand gesture recognition logic 120 is configured toidentify the hands 130 a, 130 b of the user 128 from other backgroundobjects and/or people.

In some situations, two or more people may be visible within the fieldof view of the image capture devices 118, 132. For example, smallchildren may be in the same room as the user 128. It is appreciated thatthe small children are not authorized users whose hand gestures shouldbe interpreted as commands for the media device 102. In someembodiments, the hand gesture recognition logic 120, or other suitablelogic, identifies authorized users from other viewers or other peoplewithin the field of view of the image capture devices 118, 132. Thus,hand gestures from an authorized user 128 may be identified from handgestures of other people such that commands are generated only from handgestures from the authorized user 128. In some embodiments, there may bemultiple authorized users 128. Any suitable person identification logic,such a facial recognition programs or the like, may be used to identifyan authorized user 128.

In other embodiments, the field of view of the image capture devices118, 132 may be selected so that captured images are substantiallylimited to the user 128, and/or limited to regions where the user'shands 130 a, 130 b are expected to be. Thus, background artifacts andother non-authorized individuals may be excluded from captured imagesthat are analyzed to determine the occurrence of a hand gesture commandfrom the user 128.

In some embodiments, multiple users 128 may be authorized to issue handgesture commands. Thus, a plurality of image capture devices 118, 132may be positioned and/or oriented at desired locations where authorizedusers 128 are expected to be located. Further, the field of view of suchimage capture devices 118, 132 may be selected so that captured imagesare substantially of the user 128, and/or of regions where the user'shands 130 a, 130 b are expected to be.

To facilitate an explanation of the functionality of the remote controldevice 200, the functionality of the exemplary media device 102, here aset top box, is now broadly described. One or more program contentstreams 136 are received by the program content stream interface 104.One or more tuners 104 a in the program content stream interface 104selectively tune to one of the program content streams 136 in accordancewith instructions received from the processor system 106. The programcontent stream 136 typically comprises a plurality of programsmultiplexed together. The processor system 106, based upon a request fora program of interest specified by the user 128, parses out programcontent associated with the program of interest. The program content isthen assembled into a stream of video and/or audio information which maybe saved into the program buffer 110 such that the program content canbe streamed out to the media presentation device, such as the television134, via the device interface 114. Alternatively, or additionally, theparsed out program content may be saved into the DVR 112 for laterpresentation.

In this simplified embodiment, the device interface 114 is illustratedas coupled to two exemplary media presentation devices, the television(TV) 134 and a surround-sound receiver 138. The video portion of thestreamed program content is displayed on the TV display 140 and theaudio portion of the streamed program content is reproduced as sounds bythe speakers 142.

The exemplary remote control device 200 provides a plurality ofactuators, such as buttons or the like, that the user 128 actuates toinitiate desired commands that are to be performed by or on the mediadevice 102, the connected television 134, and/or the surround-soundreceiver 138. A plurality of exemplary control buttons and theirassociated functionality are described in Table 1 below. Actuators mayinclude descriptive text and/or icons, adjacent to or on the actuator,to indicate the functionality of that particular actuator.

TABLE 1 Remote Control Device Actuators 202 Pause Button Pause controlof presented video program 204 Power Button Power the controlled device(On/Off) 206 Volume Up Increase volume of the audio 208 Volume DownDecrease volume of the audio 210 Channel Up Increment channel up by onechannel 212 Channel Down Increment channel down by one channel 214 GuideDisplay of an Electronic Program Guide (EPG) 216 Menu Display of a menuof device Functions 218 EPG Up Increment EPG active pane up by one row220 EPG Down Increment EPG active pane down by one row 222 EPG LeftIncrement EPG active pane left by one column 224 EPG Right Increment EPGactive pane right by one column 226 EPG Select Select EPG active pane228 FWD Forward presented video 230 REV Reverse presented video 232 FREWFast reverse/rewind presented video 234 FFWD Fast forward presentedvideo 236 Stop Stop current remote control device function 238 DVRRecord Record using digital video recorder (DVR) 240 Jump BackwardRewind presented video program to last marker 242 Wind Forward Windpresented video program to next marker 244 Number Buttons Buttons tospecify numerical values

It is appreciated that other actuators having other functionality notdescribed above may be included on the remote control device 200. Thefunctionality of such actuators, or combination of actuators, may beemulated by suitable hand gestures that generate corresponding handgesture commands. Further, it is appreciated that other types of remotecontrol devices that control other types of electronic devices may alsobe emulated by alternative embodiments of the hand gesture controlsystem 100.

In another example, the user 128 may wish to scroll through a displayedelectronic program guide (EPG). The image capture device 118, and/or theexternal image capture device 132, captures one or more real-time imagesof the user 128, and hence captures an image of the user's moving hand.Based upon the hand gesture, the hand gesture recognition logic 120determines that the user would like to “scroll downward” by one activeEPG pane. The command interpretation logic 126 then, based upon the handgesture of the user 128, generates a corresponding device command thatcauses a highlighted portion on the displayed EPG to scroll downward.Here, the actuator on the remote control device 200 is the EPG downbutton 220, followed by actuation of the EPG select button 226. Asubsequent hand gesture may then be used to select the highlightedprogram of interest.

FIGS. 3A-3M illustrate exemplary hand gestures of one of the user'shands 130 a, 130 b (FIG. 1) which generate exemplary hand gesturecommands. Each of the hand gestures begin at a initial location 302 ofthe user's hand. In some embodiments, the initial location 302 of theuser's hand is not relevant in identifying a hand gesture. In otherembodiments, the initial location 302 of the user's hand is predefined.

The user's hand then traverses along a path 304 and ends up at an endinglocation 306. The user's hand is held in a predefined position 308 whenthe user's hand arrives at the ending location 306.

FIG. 3A illustrates an exemplary hand gesture 310 which corresponds to apause command generated when the pause button 202 on the remote controldevice 200 is actuated. Here, the path 304 traversed by the user's handcorresponds to an upward and to the right movement. The path 304corresponds to a movement of the user's hand that is not expected to berandom. That is, the predefined path 304 is a specific movement requiredto cause the hand gesture control system 100 to generate a pause commandbased upon the hand gesture 310. Any suitable path of travel may be usedas the path 304.

The initial location 302 of the user's hand starts as some arbitrary orpredefined position, though it preferably starts in some predefinedregion of space with respect to the user 128 (not shown). When the user128 desires to initiate the exemplary pause command, the user 128 movesone of their hands generally to the initial location 302.

Then, the user 128 moves their hand along the predefined path 304. Whenthe user's hand arrives at the ending location 306, the user 128 endsthe motion. That is, the user 128 stops their hand gesture 310.

Since a series of images have been captured while the user 128 isperforming the hand gesture 310, the processor system 106, by executingthe hand gesture recognition logic 120, is able to track the movement ofthe user's hand along the path 304. At least one image shows the user'shand at the initial location 302. A later image shows the user's hand atthe ending location 306. Intervening images sequentially show themovement of the user's hand along the path 304.

In some embodiments, the position of the user's hand has a predefined,unique meaning. For example, with the hand gesture 310 corresponding toa pause command, the user's hand is extended in an outward, palm-outposition. Any suitable hand position may be predefined to correspond toa command function. Here, the outward, palm-out position at the endinglocation 306 corresponds to the pause command.

In some embodiments, the position of the user's hand at the initiallocation 302 may optionally have to be in a predefined position. Here,the user's hand is illustrated in the outward, palm-out position.However, requiring the user's hand to be in a particular position whileat the initial location 302 is optional.

Similarly, the alternative embodiments, the position of the user's handwhile traversing the path 306 may have to be held in a predefinedposition. However, requiring the user's hand to be in a particularposition while traversing the path 304 is optional.

FIG. 3B illustrates an exemplary hand gesture 312 which corresponds to astop function command generated when the stop button 236 on the remotecontrol device 200 is actuated. Here, the non-random, predefined path304 is again traversed by the user's hand. Also, the closed-hand or fistposition of the user's hand corresponds to the stop function command.The exemplary hand gesture 312 would be used, for instance, when theuser 128 has been previously performing another operation, such as on adisplayed EPG. In this exemplary scenario, the stop command generated bythe hand gesture control system 100 as a result of detecting the handgesture 312 would generate a corresponding emulated remote controldevice command to end display of the EPG.

FIG. 3C illustrates an exemplary hand gesture 314 which corresponds toan increase volume command generated when the increase volume button 206on the remote control device 200 is actuated. Here, the upward andnon-random, predefined path 304 is traversed by the user's hand. Also,the upward extended index finger position of the user's hand correspondsto the increase volume command. In this exemplary scenario, the increasevolume command generated by the hand gesture control system 100, as aresult of detecting the hand gesture 314, would generate a correspondingemulated remote control device command to increase the audio volume ofthe presented program.

FIG. 3D illustrates an exemplary hand gesture 316 which corresponds to adecrease volume command generated when the decrease volume button 208 onthe remote control device 200 is actuated. Here, the downward andnon-random, predefined path 304 is traversed by the user's hand. Also,the downward extended index finger position of the user's handcorresponds to the decrease volume command. In this exemplary scenario,the decrease volume command generated by the hand gesture control system100, as a result of detecting the hand gesture 316, would generate acorresponding emulated remote control device command to decrease theaudio volume of the presented program.

FIG. 3E illustrates an exemplary hand gesture 318 which corresponds to acommand that moves the active pane of an EPG to the right by oneposition (EPG right command). The same command is generated when the EPGright button 224 on the remote control device 200 is actuated. Here, thenon-random right hand direction of the predefined path 304 is traversedby the user's hand. Also, the extended index finger position of theuser's hand towards the right corresponds to the EPG right command. Inthis exemplary scenario, the EPG right command generated by the handgesture control system 100, as a result of detecting the hand gesture318, would generate a corresponding emulated remote control devicecommand to increment an EPG active pane right by one column on adisplayed EPG.

FIG. 3F illustrates an exemplary hand gesture 320 which corresponds toan EPG left command generated when the EPG left button 222 on the remotecontrol device 200 is actuated. Here, the non-random left hand directionof the predefined path 304 is traversed by the user's hand. Also, theextended index finger position of the user's hand towards the leftcorresponds to the EPG left command. In this exemplary scenario, the EPGleft command generated by the hand gesture control system 100, as aresult of detecting the hand gesture 320, would generate a correspondingemulated remote control device command to increment an EPG active paneleft by one column on the displayed EPG.

FIG. 3G illustrates an exemplary hand gesture 322 which corresponds to ajump backward command generated when the jump backward button 240 on theremote control device 200 is actuated. Here, the non-random counterclockwise hand direction of the predefined path 304 is traversed by theuser's hand. Also, the extended palm-out position of the user's handduring the counter clockwise movement corresponds to the jump backwardcommand. In this exemplary scenario, the jump backward command generatedby the hand gesture control system 100, as a result of detecting thehand gesture 322, would generate a corresponding emulated remote controldevice command to rewind the presented video program back to a lastmarker embedded in the program data stream.

FIG. 3H illustrates an exemplary hand gesture 324 which corresponds to awind forward command generated when the wind forward button 242 on theremote control device 200 is actuated. Here, the non-random clockwisehand direction of the predefined path 304 is traversed by the user'shand. Also, the extended palm-out position of the user's hand during theclockwise movement corresponds to the wind forward command. In thisexemplary scenario, the wind forward command generated by the handgesture control system 100, as a result of detecting the hand gesture324, would generate a corresponding emulated remote control devicecommand to wind (advance) the presented video program forward to a nextmarker embedded in the program data stream.

In some embodiments, multiple hand gestures performed in sequence willresult in generation of a hand gesture command corresponding to anactivated actuator on the remote control device 200. FIG. 3I illustratesan exemplary hand gesture 326 which corresponds to a fast reverse/rewindcommand generated when the fast rewind button 232 on the remote controldevice 200 is actuated. Here, two non-random hand gestures to the rightalong the predefined path 304 is traversed by the user's hand. Also, theclosed hand or fist position of the user's hand during the two handgestures corresponds to the fast reverse/rewind command. In thisexemplary scenario, the fast reverse/rewind command generated by thehand gesture control system 100, as a result of detecting the handgesture 326, would fast reverse/rewind the presented video program.(Further, when the user 128 wishes to stop the fast reverse/rewind ofthe presented video program, the user 128 would generate a correspondingemulated remote control device command to perform the above-describedstop function command of FIG. 3B.)

FIG. 3J illustrates an exemplary hand gesture 328 which corresponds to afast forward command generated when the fast forward button 234 on theremote control device 200 is actuated. Here, two non-random handgestures to the left along the predefined path 304 is traversed by theuser's hand. Also, the closed hand or fist position of the user's handduring the two hand gestures corresponds to the fast forward command. Inthis exemplary scenario, the fast forward command generated by the handgesture control system 100, as a result of detecting the hand gesture328, would fast forward the presented video program. (Further, when theuser 128 wishes to stop the fast forwarding of the presented videoprogram, the user 128 would generate a corresponding emulated remotecontrol device command to perform the above-described stop functioncommand of FIG. 3B.)

In some embodiments, multiple different hand gestures performed insequence will result in generation of a hand gesture commandcorresponding to a plurality of sequentially activated actuators on theremote control device 200. FIG. 3K illustrates an exemplary hand gesture330 which would generate a corresponding emulated remote control devicecommand to perform a slow motion reverse play function. Here, a pausehand gesture command is followed by a reverse direction hand gesturecommand. A slow motion reverse play function command is generated whenthe pause button 202 and the reverse button 230 on the remote controldevice 200 are sequentially actuated. Here, the two non-random handgestures along the predefined paths 304 a, 304 b are sequentiallytraversed by the user's hand. However, the traversed paths 304 a and 304b are different. The traversed path 304 a is an upwards and to the leftmovement of the user's hand. The traversed path 304 b is a short jabbingmotion to the right.

FIG. 3L illustrates an exemplary hand gesture 332 which would generate acorresponding emulated remote control device command to perform a slowmotion forward play function. Here, a pause hand gesture command isfollowed by a forward play direction hand gesture command. A slow motionforward play function command is generated when the pause button 202 andthe forward button 228 on the remote control device 200 are sequentiallyactuated. Here, the two non-random hand gestures along the predefinedpaths 304 a, 304 c are sequentially traversed by the user's hand.However, the traversed paths 304 a and 304 c are different. Thetraversed path 304 a is an upwards and to the left movement of theuser's hand. The traversed path 304 c is a short jabbing motion to theleft.

FIG. 3M illustrates an exemplary hand gesture 334 which would generate acorresponding emulated remote control device command to increase thechannel upwards by one channel function. Here, a channel up hand gesturecommand is followed by a selection hand gesture command. A singlechannel up command is generated when the channel up button 210 and theselect button 226 on the remote control device 200 are sequentiallyactuated. Here, the two non-random hand gestures along the predefinedpaths 304 a, 304 d are sequentially traversed by the user's hand.However, the traversed paths 304 a and 304 d are different. Thetraversed path 304 a is an upwards and to the left movement of theuser's hand. The traversed path 304 c is a short jabbing motion to theright.

Further, the position of the user's hand at the initial location 302during the first, channel up hand gesture has the user's index fingerextending in an upwards direction (similar to the position of the user'shand at the ending location 306). During the second hand gesture, theclosed hand or fist position of the user's hand is used (correspondingto the above described reverse command). Thus, in this example, theposition of the user's hand while at the initial location 302 isrelevant to determination of the intended command of the user 128.

Also, the position of the user's hand at the ending location 306 of thesecond hand gesture was in the closed hand or fist position. A similarclosed hand or fist position at the ending location 306 was used todefine the fast forward play hand gestures illustrated in FIG. 3I. Here,position of the user's hand (upward pointed index finger) of the firstgesture, followed by the second hand gesture in the closed hand or fistposition, defined the intended command to select a single channel upchange.

It is appreciated that, with respect to FIG. 3M, one or moreintermediate hand gestures corresponding to the first hand gesture(upward motion of the user's hand with the upward pointing index finger)may be used to indicate the number of upward channel changes. Forexample, if the user 128 made three hand gestures with an upwardspointed index finger as described, followed by the closed hand or fisthand gesture, that a device command would be generated by the handgesture control system 100 that would change the channel upwards bythree channels.

In view of the above-described hand gesture commands of FIGS. 3A-3M, itis appreciated that the position of the user's hands during a sequenceof hand gestures, and/or the shape of the predefined traversed pathsused during the sequential hand gestures, may be the same or they may bedifferent. Further, the user's hand position at the initial location 302of the traversed path 304 may be relevant to the nature of thedetermined hand gesture command. Many different predefined paths thatare to be traversed by the user's hands 130 a and/or 130 b are possible.Further, many different initial and ending hand positions are possible.Accordingly, it is appreciated that a particular hand gesture, orsequence of hand gestures, may be defined so as to emulate theactuation, or series of actuations, of any actuator(s) of the emulatedremote control device 200.

FIG. 4 is a block diagram of an electronic device 400 with an embodimentof the hand gesture control system 100. The non-limiting exemplaryembodiment 400 comprises a processor system 106, a memory 108, a signaloutput device 116, and an image capture device 118. The memory 108comprises portions for storing the hand gesture recognition logic 120,the hand gesture database 122, the remote control device (RCD) commanddatabase 124, and the command interpretation logic 126 as describedabove and illustrated in FIG. 1.

Here, the electronic device 400 is a relatively small electronic device.The electronic device 400 is placed in proximity to or coupled to acontrolled device, illustrated here as the television 134 and/or thesurround system 138. The signal output device 116 is configured tocommunicate with a receiver (not shown) in the controlled device thatreceives commands or signals from a remote control device (not shown).Thus, the electronic device 400 may be placed so that one or more handgestures from a user 128 generate emulated remote control devicecommands that emulate device commands generated by the remote controldevice that is configured to operate that particular controlled device.

Some embodiments of the electronic device 400 may be configured tooperate a variety of different controlled devices. Thus, logic isincluded in the memory 108 that allows identification of remote controldevice commands that operate the controlled device. Such commands mayreside in the RCD command database 124 (FIG. 1). Alternatively, theelectronic device 400 may be configured to control a specific controlleddevice. Thus, such an embodiment would have specific, and predefined,remote control device commands residing in the RCD command database 124.

An optional exemplary feature employed by some embodiments of the handgesture control system 100 includes an activation command feature. Aspecial hand gesture activation command must be first detected prior tothe hand gesture control system 100 issuing a device command. Thus,ordinary movement of the user 128 will not inadvertently result in ageneration of a device command. Until the activation hand gesturecommand is identified, the hand gesture control system 100 remainsdisabled so that device commands are not generated based upon detectedhand gestures. Here, an activation engagement hand gesture is stored inthe hand gesture database 122 of the memory 108. Hand gestures from theuser 128 are continuously monitored. Once the hand gesture controlsystem 100 determines that the user 128 has made the activation handgesture corresponding to the activation hand gesture command, futuredetected hand gestures may result in the generation of device commands.Some embodiments may also include a deactivation hand gesture commandfeature such that, upon detecting the user 128 making the deactivationhand gesture, the hand gesture control system 100 then becomesconfigured to disregard further hand gestures.

Another optional exemplary feature employed by some embodiments of thehand gesture control system 100 includes an engagement command feature.A special hand gesture engagement command must be first detected priorto the hand gesture control system 100 issuing a device command. Whenthe hand gesture control system 100 determines that the user 128 has theengagement hand gesture, that user 128 is monitored for a hand gesturecommand that is expected to be immediately forthcoming. Some embodimentsmay also include a disengagement hand gesture command feature such that,upon detecting the user 128 making the disengagement hand gesture, thehand gesture control system 100 then understands that the user 128 isdone making their hand gesture commands. For example, such a featurewould be used to cause display of an electronic program guide (EPG) andthen end display of the EPG when the user 128 is done navigation throughthe displayed EPG.

In some embodiments, the hand gesture control system 100 will include anoptional learning function that allows the hand gesture control system100 to recognize a particular hand gesture. Here, the hand gesturecontrol system 100 is first placed into a learning mode. For example, aspecial actuator, or a series of actuators, on the remote control device200 and/or the media device 102 may be actuated to place the handgesture control system 100 in the learning mode of operation. To start,the user 128 may enter one command, or a series of commands, byactuating the buttons or the like on their remote control device 200and/or the media device 102.

Upon conclusion of the entry of the one or more commands, the imagecapture devices 118, 132 would monitor the user 128 and determine aunique user defined hand gesture command. Then, the unique user definedhand gesture command is associated with the one or more enteredcommands, and then is saved into the hand gesture database 122.Accordingly, a particular user 128 may preprogram a series of usercommands to the hand gesture control system 100. The hand gesturecontrol system 100 “remembers” the particular, unique learned handgesture command corresponding to the series of commands.

Then, during operation, the hand gesture movement of the user 128 may becompared with the unique user defined hand gesture command associatedwith the series of remote control device commands. For example, aparticular hand gesture may be defined to change the current channelpresented on the TV 134 to a particular channel of interest. If theremote control device 200 is used to change channels, several differentbuttons or the like may need to be actuated on their remote controldevice 200. On the other hand, a single unique learned hand gesturecommand may be used to generate the series of commands.

In an exemplary embodiments, a calibration screen may be displayed sothat the user 128 can confirm the remote control device command(s) thatwill be used to create a learned user defined hand gesture command. Thecalibration screen may display a textual listing, and/or an iconiclisting, of the remote control device commands. Also, the calibrationscreen may show a video of the user's unique hand gesture, and/orgenerate a cartoon-like video of the user's hand gesture. Furthertesting and confirmation of the user defined hand gesture command may becontrolled via the calibration screen, which may be configured as agraphical user interface type menu.

For example, a user defined hand gesture ending with two upwardlyextended fingers on the user's right hand 130 a and/or on the user'sleft hand 130 b may be understood to correspond to a learned userdefined hand gesture command to change the channel to a first predefinedchannel. Similarly, a hand gesture ending with three upwardly extendedfingers may be understood to correspond to a learned user defined handgesture command to generate a corresponding emulated remote controldevice command to change the channel to a second predefined channel.

Some embodiments may analyze hand gestures of the user 128 based uponcaptured image information provided by more than one image capturedevices. Upon determination of a hand gesture movement captured by afirst image capture device, image information from a second imagecapture device, or a plurality of other image capture devices, is usedto verify the hand gesture command. That is, hand gesture commands thatare determined from multiple images are compared, and then issued asdevice commands when the comparison verifies that the user 128 has madea particular hand gesture command.

It should be emphasized that the above-described embodiments of the handgesture control system 100 are merely possible examples ofimplementations of the invention. Many variations and modifications maybe made to the above-described embodiments. All such modifications andvariations are intended to be included herein within the scope of thisdisclosure and protected by the following claims.

What is claimed is:
 1. A media device, comprising: a program contentstream interface configured to receive a program content streamcomprising media content; an image capture device configured to capturea series of images of a user, and configured to generate imageinformation corresponding to the series of captured images; a memorystoring a plurality of predefined hand gesture commands, each of thepredefined hand gesture commands uniquely corresponding to at least oneof a plurality of commands associated with a remote control device,wherein the remote control device is configured to control at least oneof the media device and an external device; a processor systemcommunicatively coupled to the image capture device and the memory,wherein the processor system is configured to: receive the imageinformation from the image capture device, identify at least one handgesture made by the user from the received image information, determinea hand gesture command based upon the identified hand gesture, comparethe determined hand gesture command with the plurality of predefinedhand gesture commands, identify a matching hand gesture command from theplurality of predefined hand gesture commands corresponding to thedetermined at least one hand gesture command, determine whether the handgesture command corresponds with operation of one of the media deviceand the external device, if the hand gesture command corresponds withoperation of the media device, modifying output of the media contentreceived by the program content stream interface based on the handgesture command, and if the hand gesture command corresponds with theoperation of the external device, generating an emulated remote controldevice command based upon the identified matching hand gesture commandthat is configured to control the operation of the media device; and asignal output device communicatively coupled to the processor system andconfigured to wirelessly communicate the emulated remote control devicecommand from the media device to the external device, wherein thecommunicated emulated remote control device command is the same as acorresponding wireless command that would be communicated from theremote control device.
 2. The media device of claim 1, wherein thesignal output device comprises an infrared (IR) emitter configured toemit the emulated remote control device command as an IR command that isreceivable by the external device.
 3. The media device of claim 1,wherein the signal output device comprises a radio frequency (RF)emitter configured to emit the emulated remote control device command asa RF command that is receivable by the external device.
 4. The mediadevice of claim 1, wherein the signal output device configured tocommunicate the emulated remote control device command to at least onemedia presentation device using a communication media that is used bythe remote control device.
 5. The media device of claim 1, wherein theimage capture device is an external image capture device that iscommunicatively coupled to the processor system.
 6. The media device ofclaim 1, wherein the image capture device is included in the mediadevice.
 7. The media device of claim 1, wherein the determined handgesture command causes the signal output device to output the emulatedremote control device command as a serial output of a plurality ofcontrol signals, wherein each of the plurality of output control signalsare the same as a corresponding plurality of remote control devicecommands that are configured to control a same operation of the mediadevice.
 8. The media device of claim 1, wherein the at least one handgesture is a first hand gesture, and wherein the series of captureimages includes at least the first hand gesture and a second handgesture, wherein the processor system is configured to: identify thesecond hand gesture made by the user from the received imageinformation, and determine a hand gesture command based upon theidentified first hand gesture and the identified second hand gesture. 9.An apparatus, comprising: an image capture device configured to capturea series of images of a user, and configured to generate imageinformation corresponding to the series of captured images; a memorystoring a plurality of predefined hand gesture commands, each of thepredefined hand gesture commands uniquely corresponding to at least oneof a plurality of commands associated with a media device; a processorsystem communicatively coupled to the image capture device and thememory, wherein the processor system is configured to: receive the imageinformation from the image capture device, identify at least one handgesture made by the user from the received image information, determinea hand gesture command based upon the identified at least one handgesture, compare the determined hand gesture command with the pluralityof predefined hand gesture commands, identify a matching hand gesturecommand from the plurality of predefined hand gesture commandscorresponding to the determined hand gesture command, generate anemulated remote control device command based upon the identifiedmatching hand gesture command that is configured to control operation ofthe media device; and a signal output device configured to communicatethe emulated remote control device command wirelessly to an externaldevice.
 10. The apparatus of claim 9, wherein the signal output devicecomprises an infrared (IR) emitter configured to generate an IR commandfor the external device.
 11. The apparatus of claim 9, wherein thesignal output device comprises a radio frequency (RF) emitter configuredto emit the emulated remote control device command as a RF command thatis receivable by the external device.
 12. The apparatus of claim 9,wherein the determined hand gesture command causes the signal outputdevice to output the emulated remote control device command as a serialoutput of a plurality of signals, wherein each of the plurality ofoutput signals are the same as a corresponding plurality of remotecontrol device commands that are configured to control a same operationof the external device.
 13. The apparatus of claim 9, wherein theexternal device is another media device configured to receive at leastone of a broadcasted program content stream with media content therein,wherein the emulated remote control device command is configured tocontrol the media device.
 14. The apparatus of claim 9, wherein theexternal device is another media device configured to receive at leastone of a broadcasted program content stream with media content therein,wherein the emulated remote control device command is configured tocontrol a presentation device communicatively coupled to the mediadevice, and wherein in response to receiving the emulated remote controldevice command at the media device, the media device generates andcommunicates commands to the presentation device to control presentationof media content by the presentation device in accordance with theemulated remote control device command.
 15. A method to generatecommands based upon hand gestures made by a user, wherein the commandsemulate device commands generated by a remote control device, the methodcomprising: generating image information from a series of capturedimages; identifying a hand gesture made by the user from the generatedimage information; comparing the identified hand gesture with aplurality of predefined hand gesture commands to identify a hand gesturecommand that matches one of the plurality of predefined hand gesturecommands; generating an emulated remote control device command basedupon the identified matching hand gesture command; and communicating theemulated remote control device command wirelessly to a media device,wherein the media device is controlled based upon the wirelesslycommunicated emulated remote control device command.
 16. The method ofclaim 15, wherein identifying the hand gesture made by the user from theimage information comprises: identifying a serial plurality of handgestures made by the user; determining a first hand gesture command froma first portion of the series of captured images of the user;determining a second hand gesture command from a second portion of theseries of captured images of the user, wherein the second portion of theseries of captured images is different from the first portion of theseries of captured images; comparing the determined first hand gesturecommand with the plurality of predefined hand gesture commands toidentify a first matching hand gesture command from the plurality ofpredefined hand gesture commands; comparing the determined second handgesture command with the plurality of predefined hand gesture commandsto identify a second matching hand gesture command from the plurality ofpredefined hand gesture commands; and generating the emulated remotecontrol device command based upon the identified first matching handgesture command and the second matching hand gesture command.
 17. Themethod of claim 16, wherein the emulated remote control device commandcontrols a display based on an electronic program guide (EPG) displayedon the display, and further comprising: determining a subsequent handgesture command from a subsequent second series of captured images ofthe user; comparing the determined subsequent hand gesture command withthe plurality of predefined hand gesture commands to identify asubsequent matching hand gesture command from the plurality ofpredefined hand gesture commands, wherein the subsequent matching handgesture command corresponds to a remote control device command thatcauses movement of an active pane of the EPG; generating a subsequentemulated remote control device command based upon the identifiedsubsequent matching hand gesture command; and controlling the displaybased upon the subsequent emulated remote control device command suchthat the active pane of the EPG is moved from a first position to asecond position.
 18. The method of claim 16, further comprising:determining an enablement hand gesture command from a first one of theserial plurality of hand gestures made by the user, wherein a subsequenthand gesture command determined from at least a second one of theplurality of hand gestures is used to generate the emulated remotecontrol device command, and wherein the emulated remote control devicecommand is determined only after the enablement hand gesture command isdetermined.
 19. The method of claim 16, further comprising: determininga disablement hand gesture command from a last one of the serialplurality of hand gestures made by the user, wherein further determineduser hand gestures are disregarded, wherein the emulated remote controldevice command is based upon other ones of the serial plurality of handgestures made by the user.
 20. The method of claim 15, whereincommunicating the emulated remote control device command comprisescommunicating the emulated remote control device command as one of aradio frequency (RF) signal and an infrared (IR) signal receivable bythe media device.